|
Prepared for: Federal Motor Carrier Safety Administration Office of Research and Analysis Analysis Division, MC-RRA 1200 New Jersey Avenue, S.E. Washington, DC 20590 |
Prepared by: John A. Volpe National Transportation Systems Center Office of Surface Transportation Programs Motor Carrier Safety Division, RTV-3E Kendall Square Cambridge, MA 02142 |
This report documents the methodology and results from the Federal Motor Carrier Safety Administration’s (FMCSA) Compliance Review (CR) Effectiveness Model. This model measures the effectiveness of one of the key safety programs of the FMCSA, the compliance review program. The model was developed for the FMCSA by the Research and Innovative Technology Administration's (RITA) John A. Volpe National Transportation Systems Center (the Volpe Center) in Cambridge, MA. This work is part of an effort to assess the effectiveness of the FMCSA’s principal safety programs. The work also addresses the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the results of their programs as part of the budget cycle process.
The CR Effectiveness Model is one of two models that provide a baseline of the effectiveness of FMCSA safety programs through the use of standard safety performance measures. This baseline allows the FMCSA to judge the relative performance of its programs on a periodic basis by reflecting the changes in benefits resulting from each program. The results of these analyses are also intended to provide a basis for FMCSA resource allocation and budgeting decisions that will more closely optimize the effectiveness and efficiency of its motor carrier safety programs.
In addition to the CR Effectiveness Model, the Intervention Model has been developed to measure the effectiveness of and estimate benefits resulting from roadside inspection and traffic enforcement activities. These two models have been developed to estimate the benefits of these FMCSA safety programs in terms of crashes avoided, lives saved, and injuries avoided.
This project is managed by the Analysis Division in the FMCSA's Office of Research and Analysis. The Volpe Center project manager is Kevin Gay of the Motor Carrier Safety Division. The analysis was performed by Jon Ohman with assistance from Kevin Gay and Julie Nixon, all of the Volpe Center. Technical support was provided by Richard Nguyen of the Volpe Center and Leon Parkin of Chenega Advanced Solutions & Engineering (CASE), LLC, under contract to the Volpe Center. Olu Ajayi of the FMCSA’s Analysis Division deserves special thanks for his assistance in obtaining data that were used in the implementation of the model.
APPENDIX A: CALCULATION OF PRE-CR AND POST-CR AVERAGE CRASH RATES FOR CONTROL GROUP
2-1. Compliance Review Effectiveness Model
2-2. Timeline for a Carrier with a Compliance Review on August 15, 2005
3-1. Results of Implementation of Model by Carrier Size
3-2. Results of Implementation of Model by State of Domicile of Carrier
3-3. Results of Implementation of Model by Carrier Safety Status
3-4. Results of Implementation of Model by Type of Planned Course of Action
This report documents the methodology and results from a model that measures the effectiveness of one of the key safety programs of the Federal Motor Carrier Safety Administration (FMCSA), the compliance review (CR) program. The research was conducted by the Research and Innovative Technology Administration's (RITA) John A. Volpe National Transportation Systems Center (the Volpe Center) in Cambridge, MA under a project plan agreement with the FMCSA. The work on the FMCSA Safety Program Effectiveness Measurement Project addresses the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the results of their programs as part of the budget cycle process.
This report describes the methodology of the Compliance Review Effectiveness Model and presents the results of the implementation of the model for carriers receiving CRs in fiscal year (FY) 2005. The benefits of the compliance review program are calculated in terms of crashes avoided, lives saved, and injuries avoided.
The on-site compliance review is perhaps the single greatest resource-consuming activity of the FMCSA. Thousands of CRs are conducted each year. In FY 2005, federal and state enforcement personnel conducted over 11,000 CRs on individual motor carriers. It is intended that through education, heightened safety regulation awareness, and the enforcement effects of the CR, carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce the number and severity of crashes in which they are involved.
The CR Effectiveness Model was developed to determine the effectiveness of the CR program. The model measures the direct impact of compliance reviews on carriers that received CRs, but not "deterrent" effects (i.e., the "threat" of having a CR) on carriers that did not actually receive CRs. The model is based on the individual and cumulative "before and after" changes in the safety performance of carriers that received CRs in a given year. The model compares a motor carrier's crash rate in the 12 months following an on-site compliance review to its crash rate in the 12 months prior to that review. The model uses (1) crash data reported by the states and (2) power unit data reported by carriers or obtained during CRs, to calculate both the before-CR and after-CR crash rates.
To eliminate the effects of underlying trends occurring in the general carrier population, a control group of carriers is used. This Control Group consists of all carriers that did not receive CRs during the year in question. Any change in the average crash rate of the Control Group must be due to factors affecting the entire carrier population. Thus, the change in the average crash rate of the Control Group is calculated and then subtracted from the change in the average crash rate of the carriers that received CRs in the year in question. The difference resulting from this calculation represents the change in the average crash rate of the carriers that received CRs in the year in question that was solely the result of the CRs.
All previous implementations of the model were on a calendar year (CY) basis. That is, the model has been used to estimate benefits only for carriers with CRs conducted in CY 2002, 2003, and 2004. Beginning with this report, the model will be implemented on a fiscal year (FY) basis to align the activities of the CR program with the program’s funding cycle. It will now be possible to link the results of the CRs conducted during a given fiscal year with the funding for the CR program for that fiscal year.
The model succeeded the Compliance Review Impact Analysis Model, which was used to estimate the benefits for carriers with CRs in CY 1998, 1999, 2000, and 20011. The estimates produced by the CR Effectiveness Model will establish new benchmarks and are not directly comparable to the estimates produced by the CR Impact Assessment Model.
The CR Effectiveness Model was implemented for carriers with CRs in FY 2005 to estimate the number of crashes (and associated fatalities and injuries) avoided in the first year following the reviews, i.e., FY 2005-2006. Table ES-1 shows these benefits, as well as the benefits that were estimated to have occurred (1) in CY 2002-2003 for carriers with CRs in CY 2002, (2) in CY 2003-2004 for carriers with CRs in CY 2003, and (3) in CY 2004-2005 for carriers with CRs in CY 2004.
| Model Implementation for Motor Carriers with CRs in: | CY 2002 | CY 2003 | CY 2004 | FY 2005 |
|---|---|---|---|---|
| Compliance reviews conducted | 12,139 | 11,086 | 10,671 | 11,431 |
Motor carriers that received compliance reviews and:
|
9,172 | 8,587 | 8,042 | 8,941 |
| Estimated percentage reduction in average crash rate due to compliance reviews | 12.6 | 17.6 | 21.1 | 16.3 |
| Model Results (i.e., Benefits) Estimated for: | CY 2002-2003 | CY 2003-2004 | CY 2004-2005 | FY 2005-2006 |
| Crashes avoided | 1,426 | 2,276 | 2,720 | 2,306 |
| Fatal crashes | 53 | 77 | 92 | 79 |
| Injury crashes | 677 | 1,038 | 1,186 | 982 |
| Towaway crashes | 696 | 1,161 | 1,442 | 1,245 |
| Lives saved | 62 | 90 | 107 | 92 |
| Injuries avoided | 1,087 | 1,651 | 1,889 | 1,561 |
To further assess the effectiveness of the compliance review program, the results of the implementation of the model were broken out by carrier size (i.e., number of power units), by the state of domicile of the carrier, by carrier safety status (i.e., the carrier's SafeStat2 category before receiving its CR in FY 2005), and by the planned course of action (i.e., enforcement or no enforcement) for the carrier.
During the 1980s, Congress passed several acts intended to strengthen motor carrier safety regulations. This led to the implementation of safety-oriented programs both at the federal and state levels. The Surface Transportation Assistance Act of 1982 established the Motor Carrier Safety Assistance Program, a grants in aid program to states to conduct roadside inspection and traffic enforcement programs aimed at commercial motor vehicles. The 1984 Motor Carrier Safety Act directed the U.S. Department of Transportation (U.S. DOT) to establish safety fitness standards for carriers. The U.S. DOT, in conjunction with the states, implemented the Motor Carrier Safety Assistance Program (MCSAP) to fund the roadside inspection and traffic enforcement programs and the safety fitness determination process and rating system (based on on site safety audits called compliance reviews).
It is expected that a major benefit of these programs has been and will continue to be an improved level of safety in the operation of commercial motor vehicles. Previously, however, there was no means to measure the benefits and effectiveness of these programs. The Safety Program Effectiveness Measurement Project was established to identify major functions and operations (programs) associated with the FMCSA mission and to develop results-oriented performance measures for those functions and operations, as called for in the Government Performance and Results Act (GPRA) of 1993.
Program evaluation should be viewed as a continuous management process that encourages the organization to reflect periodically upon how it is implementing its programs. Program effectiveness should be reassessed in light of the mission, available resources, changing requirements, political climate, technological change, public demands, and costs. Periodic review of the results of the evaluations will ensure that the activities are working, i.e., that they are delivering what was promised. This report is intended to satisfy the desire of the FMCSA to verify the effectiveness of one of its motor carrier safety programs, the compliance review program. The immediate objective of this effort is to measure how much of an impact the safety program activities have on avoiding crashes involving motor carriers and reducing resulting injuries and fatalities.
One of the main objectives of the Safety Program Effectiveness Measurement Project is to provide a baseline of the effectiveness of the selected programs through the use of standard safety performance measures. This baseline allows the FMCSA to judge the relative performance of its programs on a periodic basis by reflecting the benefits resulting from each program. The results of these analyses are intended to provide a basis for FMCSA resource allocation and budgeting decisions that will more closely optimize the effectiveness and efficiency of its motor carrier safety programs.
The scope of this overall effort is limited to the major identifiable operational FMCSA programs and their effectiveness in reducing crashes and avoiding injuries and fatalities. Currently the Safety Program Effectiveness Measurement Project includes the compliance review, roadside inspection, and traffic enforcement activities and programs performed and supported by the FMCSA. Two models have been developed to estimate the benefits of these programs: the Compliance Review Effectiveness Model and the Intervention Model (for roadside inspections and traffic enforcements). The benefits of these programs are calculated in terms of crashes avoided, lives saved, and injuries avoided.
An objective of the project is to continue to improve these models and update the results on a recurring basis. The models will serve the program-specific requirement to measure program effectiveness as well as the broader function of supporting annual budget requirements and helping to determine the best resource allocation among program elements.
This report describes the methodology of the Compliance Review Effectiveness Model and presents the results of the implementation of the model for carriers receiving compliance reviews (CRs) in FY 2005, including estimates of crashes avoided by carrier size, state of domicile, carrier safety status, and planned course of action.
All previous implementations of the model were on a calendar year (CY) basis. That is, the model has been used to estimate benefits only for carriers with CRs conducted in CY 2002, 2003, and 2004. Beginning with this report, the model will be implemented on a fiscal year (FY) basis to align the activities of the CR program with the program's funding cycle. It will now be possible to link the results of the CRs conducted during a given fiscal year with the funding for the CR program for that fiscal year.
The model succeeded the Compliance Review Impact Assessment Model, which was used to estimate the benefits for carriers with CRs in CY 1998, 1999, 2000, and 20013. The results from the two models are not directly comparable, because the estimates produced by the CR Effectiveness Model will establish new benchmarks, which may differ from the level of the estimates produced by the CR Impact Assessment Model.
The on-site compliance review (CR) is perhaps the single greatest resource-consuming activity of the FMCSA. Thousands of CRs are conducted each year. In FY 2005, federal and state enforcement personnel conducted over 11,000 CRs on individual motor carriers. In addition to actually conducting CRs, the FMCSA invests in: extensive analysis of the requirements of the Federal Motor Carrier Safety Regulations (FMCSR), enhancements to the design of the CR to better assess safety performance and compliance with the FMCSR, continued safety investigator training, enhancements to prioritization methodologies such as SafeStat4 to determine what carriers should receive CRs, and enhancements to information systems to report and store the results of the CRs that are conducted.
When performing CRs, FMCSA and state safety investigators spend many hours examining the safety records of individual motor carriers to assess their compliance and safety performance. The investigators also discuss their findings with the carriers' safety managers to improve understanding of their safety programs. After a review is completed, the carrier is assigned a safety rating (i.e., satisfactory, conditional, or unsatisfactory). If serious violations are discovered, an enforcement case is initiated and a fine may be imposed. The CR results are also incorporated, with other safety data (i.e., crashes, roadside inspection results, moving violations, and closed enforcement cases), into SafeStat to reassess the carrier's safety status. It is intended that through education, heightened safety regulation awareness, and the enforcement effects of the CR, carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce the number and severity of crashes in which they are involved.
The CR Effectiveness Model was developed to determine the effectiveness of the CR program. The model measures the direct impact of compliance reviews on carriers that received CRs, but not "deterrent" effects (i.e., the "threat" of having a CR) on carriers that did not actually receive CRs. In addition, the model was developed to estimate only the benefits that occur in the 12 months following a CR. The model is based on the individual and cumulative "before and after" changes in the safety performance of carriers that have received CRs. The model compares a motor carrier's crash rate in the 12 months following an on-site compliance review to its crash rate in the 12 months prior to that review. The model uses (1) crash data reported by the states and (2) power unit data obtained during CRs or from updated Form MCS-150 information submitted by carriers, to calculate both the before-CR and after-CR crash rates. The data are stored in the FMCSA’s Motor Carrier Management Information System (MCMIS).
A diagram of the CR Effectiveness Model, as implemented for carriers with CRs in fiscal year (FY) 2005, is shown in Figure 2-1. The model estimates the number of crashes (and associated fatalities and injuries) avoided in the 12 months following the CRs. Thus, the benefits from the CRs conducted in FY 2005 occurred in both FY 2005 and FY 2006.
A step-by-step description of the implementation procedure follows. The step numbers (shown in parentheses) correspond to the numbers in parentheses in the diagram.
| Model Implementation for Motor Carriers with CRs in: | CY 2002 | CY 2003 | CY 2004 | FY 2005 |
|---|---|---|---|---|
| Compliance reviews conducted | 12,139 | 11,086 | 10,671 | 11,431 |
Motor carriers that received compliance reviews and:
|
9,172 | 8,587 | 8,042 | 8,941 |
| Estimated percentage reduction in average crash rate due to compliance reviews | 12.6 | 17.6 | 21.1 | 16.3 |
| Model Results (i.e., Benefits) Estimated for: | CY 2002-2003 | CY 2003-2004 | CY 2004-2005 | FY 2005-2006 |
| Crashes avoided | 1,426 | 2,276 | 2,720 | 2,306 |
| Fatal crashes | 53 | 77 | 92 | 79 |
| Injury crashes | 677 | 1,038 | 1,186 | 982 |
| Towaway crashes | 696 | 1,161 | 1,442 | 1,245 |
| Lives saved | 62 | 90 | 107 | 92 |
| Injuries avoided | 1,087 | 1,651 | 1,889 | 1,561 |
The results of the implementation of the model were broken out by carrier size (i.e., number of power units), by the state of domicile of the carrier, by carrier safety status (i.e., the carrier's SafeStat category before receiving its CR in FY 2005), and by the planned course of action (i.e., enforcement or no enforcement).
The results of these analyses revealed the types of carriers that will most likely respond positively to CRs. By focusing on carriers that are likely to respond positively to CRs, the effectiveness of the compliance review program may be improved. Alternative treatment approaches may be suggested for carriers that are at risk, but will most likely not respond positively to CRs.
The sums of the estimates of crashes avoided by power unit group, state of domicile, SafeStat category group, and planned course of action group did not equal the estimate of 2,306 crashes avoided that was obtained in Section 2.3. Therefore, the estimates were prorated to sum to this number. The estimated numbers of crashes avoided, the post-CR average crash rates, and the percent changes in the average crash rates shown in Tables 3-1, 3-2, 3-3, and 3-4 were all derived using this prorating procedure.
The results of the implementation of the model were broken out by carrier size as measured by the number of power units at the time of the CR, i.e., the number of pre-CR power units.
Table 3-1 shows the results of the implementation of the model for the four power unit groups:
Table 3-1 shows, for each power unit group, the number of carriers in the group that received CRs in FY 2005, the pre-CR average crash rate, the adjusted post-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-1 also shows, for each power unit group, the estimated number of crashes avoided as a result of the CRs.
| Number of Pre-CR Power Units |
Number of Carriers with CRs in FY 2005 |
Pre-CR Average Crash Rate* |
Post-CR Average Crash Rate* |
Percent Change in Average Crash Rate |
Estimated Number of Crashes Avoided in FY 2005-2006 |
|---|---|---|---|---|---|
| 1 - 5 | 4,064 | 10.832 | 5.880 | –45.7 | 583 |
| 6 - 20 | 3,097 | 7.434 | 5.202 | –30.0 | 778 |
| 21-100 | 1,430 | 5.906 | 4.912 | –16.8 | 607 |
| ≥101 | 350 | 4.908 | 4.661 | –5.0 | 338 |
| All Carriers | 8,941 | 5.785 | 4.842 | –16.3 | 2,306 |
* - Crashes per 100 power units
The reduction in the average crash rate was inversely related to the size of the carrier, i.e., the larger the carrier, the smaller the crash rate reduction. The reduction in the average crash rate ranged from 45.7 percent for carriers with 1-5 power units to 5.0 percent for carriers with 101 or more power units.
Carriers with 6-20 power units had the largest number of crashes avoided due to the program (778), followed by carriers with 21-100 power units (607). The carriers with 1-5 power units had only the third highest number of crashes avoided (583), despite having the largest crash rate reduction. This result was a consequence of the distribution of power units. The carriers with 1 5 power units accounted for 45.5 percent of the carriers in the analysis, but only 4.8 percent of the post-CR power units.
The results of this analysis are consistent with (1) the results of the analyses of data from the implementations of the model for carriers with CRs in CY 2002, CY 2003, and CY 20047 and (2) the results of analyses of data from the implementations of the previous model, the Compliance Review Impact Assessment Model.8.
Table 3-2 shows the results of the implementation of the model broken out by the carrier’s state of domicile. For a state's results to be published in the table, it had to have at least 50 carriers with CRs in FY 2005. Five states9, the District of Columbia, and Puerto Rico did not meet this requirement. Their data were combined and are shown in the row labeled "Other S. and P." (Other States and Possessions). Since there were not enough Canadian or Mexican carriers receiving CRs in FY 2005 to summarize at the province/state level, these results were summarized at the national level.
| State/Country of Domicile | Number of Carriers with CRs in FY 2005 |
Pre-CR Average Crash Rate* |
Post-CR Average Crash Rate* |
Percent Change in Average Crash Rate |
Estimated Number of Crashes Avoided in FY 2005-2006 |
|---|---|---|---|---|---|
| Alabama | 263 | 7.480 | 5.860 | –21.6 | 91 |
| Arizona | 149 | 6.252 | 5.127 | –18.0 | 276 |
| Arkansas | 148 | 9.248 | 6.736 | –27.2 | 56 |
| California | 333 | 4.850 | 4.134 | –14.8 | 56 |
| Colorado | 186 | 3.448 | 2.977 | –13.6 | 28 |
| Connecticut | 55 | 5.418 | 2.969 | –45.2 | 27 |
| Florida | 232 | 5.937 | 4.760 | –19.8 | 67 |
| Georgia | 441 | 7.718 | 5.514 | –28.6 | 160 |
| Idaho | 99 | 6.975 | 5.268 | –24.5 | 25 |
| Illinois | 310 | 7.713 | 5.635 | –26.9 | 174 |
| Indiana | 286 | 5.188 | 4.408 | –15.1 | 107 |
| Iowa | 92 | 6.597 | 7.250 | +9.9 | (49) |
| Kansas | 401 | 4.511 | 3.961 | –12.2 | 26 |
| Kentucky | 204 | 5.877 | 4.313 | –26.6 | 56 |
| Louisiana | 123 | 7.221 | 5.627 | –22.1 | 31 |
| Maine | 54 | 5.924 | 5.202 | –12.2 | 3 |
| Maryland | 76 | 3.156 | 2.770 | –12.2 | 6 |
| Massachusetts | 70 | 3.941 | 3.894 | –1.2 | 1 |
| Michigan | 154 | 7.607 | 6.917 | –9.1 | 40 |
| Minnesota | 323 | 5.172 | 4.487 | –13.3 | 58 |
| Mississippi | 148 | 6.715 | 6.070 | –9.6 | 32 |
| Missouri | 275 | 5.821 | 4.786 | –17.8 | 111 |
| Montana | 59 | 5.773 | 3.701 | –35.9 | 25 |
| Nebraska | 113 | 6.237 | 5.074 | –18.6 | 41 |
| Nevada | 124 | 3.935 | 3.200 | –18.7 | 9 |
| New Hampshire | 60 | 4.280 | 3.200 | –25.2 | 6 |
| New Jersey | 247 | 5.921 | 5.726 | –3.3 | 15 |
| New Mexico | 73 | 3.565 | 3.832 | +7.5 | (5) |
| New York | 143 | 5.863 | 4.749 | –19.0 | 31 |
| North Carolina | 310 | 8.710 | 6.537 | –25.0 | 61 |
| North Dakota | 118 | 4.696 | 3.336 | –29.0 | 24 |
| Ohio | 540 | 5.195 | 4.693 | –9.7 | 72 |
| Oklahoma | 124 | 5.333 | 5.023 | –5.8 | 10 |
| Oregon | 53 | 3.491 | 2.519 | –27.8 | 10 |
| Pennsylvania | 76 | 3.842 | 3.974 | +3.4 | (3) |
| South Carolina | 176 | 5.973 | 6.251 | +4.6 | (7) |
| South Dakota | 65 | 3.181 | 2.617 | –17.7 | 6 |
| Tennessee | 266 | 7.519 | 5.902 | –21.5 | 143 |
| Texas | 647 | 5.248 | 4.344 | –17.2 | 228 |
| Utah | 196 | 5.058 | 4.381 | –13.4 | 42 |
| Virginia | 68 | 4.238 | 2.587 | –38.9 | 23 |
| Washington | 296 | 3.765 | 3.248 | –13.7 | 28 |
| West Virginia | 96 | 8.704 | 4.387 | –49.6 | 49 |
| Wisconsin | 230 | 5.970 | 5.519 | –7.5 | 32 |
| Wyoming | 90 | 4.255 | 2.317 | –45.6 | 17 |
| Other S. & P.† | 99 | 3.479 | 1.681 | –51.7 | 33 |
| Canada | 133 | 5.323 | 3.983 | –25.2 | 34 |
| Mexico | 117 | 0.393 | 0.355 | –9.7 | 0 |
| Total | 8,941 | 5.785 | 4.842 | –16.3 | 2,306 |
* — Crashes per 100 power units
† – Other States & Possessions: Alaska, Delaware, District of Columbia, Hawaii, Puerto Rico, Rhode Island, and Vermont
Table 3-2 shows, for each state (or country), the number of carriers that received CRs in FY 2005, the pre-CR average crash rate, the adjusted post-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-2 also shows, for each state (or country), the estimated number of crashes avoided as a result of the CRs. (Note: A number in parentheses indicates an increase in the number of crashes.)
Table 3-2 shows that two states, Arizona (276) and Texas (228), had more than 200 crashes avoided in FY 2005-2006 due to CRs performed in FY 2005. Five other states (Illinois, Georgia, Tennessee, Missouri, and Indiana) each had more than 100 crashes avoided. Four states showed increases in the number of crashes in FY 2005-2006 by carriers that received CRs in FY 2005.
There are several factors that affect the state estimates of crashes avoided. The equation that is used to calculate the number of crashes avoided consists of three factors: the pre-CR average crash rate, the percentage reduction in the average crash rate due to the CRs, and the number of post-CR power units. The states with the largest numbers of crashes avoided are usually among the states with the highest numbers of post-CR power units, which is a function of the number of carriers receiving CRs. The more carriers in a state that receive reviews, the greater the number of post-CR power units that results, which increases the potential for a large number of crashes to be avoided. For example, Indiana had a reduction in its average crash rate of only 15.1 percent, but had 107 crashes avoided because it had 286 carriers with CRs in FY 2005. On the other hand, Connecticut had a reduction in its average crash rate of 45.2 percent, but had only 27 crashes avoided because it had only 55 carriers with CRs in FY 2005.
Another factor that influenced the state results was the proportion of the carriers with zero crashes in the pre-CR period in each state that received CRs in FY 2005. Of the total of 8,941 carriers that received reviews in FY 2005, 5,219, or 58.4 percent, had pre-CR crash rates of zero. Thus, the crash rates of these carriers could either stay the same or increase, but not decrease. If a state had an especially high percentage of these carriers, it would make it difficult for that state’s average crash rate to decrease significantly.
In addition, the relatively low number of carriers in each state that received CRs in FY 2005 makes the state results subject to the influence of a few large carriers, i.e., carriers with large numbers of power units. As shown in Table 3-1, there were 350 carriers with 101 or more power units that received CRs in FY 2005. While these carriers made up only 3.9 percent of the 8,941 carriers that were analyzed, they accounted for 56.0 percent of the total number of post-CR power units. Thus, the data from one or two large carriers could greatly affect an individual state’s results.
One of the primary methods of prioritizing carriers for CRs is to use SafeStat results. Carriers are assessed in four Safety Evaluation Areas (SEAs): Accident, Driver, Vehicle, and Safety Management. Carriers are placed in SafeStat categories if they are found to be deficient in one or more SEAs. Carriers with the most extensive deficiencies are placed in Categories A and B and are assigned the highest priority for CRs, followed by carriers in Category C, carriers in Categories D-G, and finally, carriers not in any category (i.e., carriers not deficient in any SEAs).
The purpose of the analysis in this section is to determine the impact of carrier safety status prior to CRs on crash rate reduction after the CRs. In other words, determine if carriers with the highest priority for CRs show the greatest improvement, i.e., the largest crash rate reduction, following their CRs.
The results of the CR Effectiveness Model were broken out by SafeStat category group based on each carrier's SafeStat category prior to receiving its FY 2005 CR. Table 3-3 shows, for each SafeStat category group, the number of carriers in the group that received CRs in FY 2005, the pre-CR average crash rate, the adjusted post-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-3 also shows, for each SafeStat category group, the estimated number of crashes avoided as a result of the CRs.
| SafeStat Category Group |
Number of Carriers with CRs in FY 2005 |
Pre-CR Average Crash Rate* |
Post-CR Average Crash Rate* |
Percent Change in Average Crash Rate |
Estimated Number of Crashes Avoided in FY 2005-2006 |
|---|---|---|---|---|---|
| A-B | 3,120 | 8.767 | 6.412 | –26.9 | 1,995 |
| C | 881 | 4.317 | 4.446 | +3.0 | (24) |
| D-G | 1,948 | 4.921 | 4.338 | –11.8 | 354 |
| None | 2,992 | 3.616 | 3.640 | +0.7 | (19) |
| All Carriers | 8,941 | 5.785 | 4.842 | –16.3 | 2,306 |
Carriers in Categories A and B, the carriers with the highest priority for CRs, had the highest pre-CR average crash rate as well as the greatest percent reduction in their average crash rate. Their post-CR average crash rate showed a decrease of 26.9 percent. The carriers in this group accounted for 1,995 of the 2,306 crashes avoided in FY 2005-2006. Carriers in Categories D-G showed a decrease of 11.8 percent in their average crash rate and had 354 crashes avoided.
Carriers in Category C showed an increase of 3.0 percent in their average crash rate, while carriers not in any SafeStat category showed an increase of 0.7 percent in their average crash rate.
The results of the implementation of the model were also broken out by the course of action planned by the FMCSA for the carrier following its FY 2005 CR. A carrier with prosecution or an out-of-service order indicated as the planned course of action was classified as an "enforcement" carrier. A carrier with only compliance monitoring indicated as the planned course of action was classified as a "non-enforcement" carrier.
It should be noted that these courses of action are the ones that were anticipated by the FMCSA at the conclusions of the CRs that the carriers received in FY 2005, and may be different from the actions that were actually taken. The data in the MCMIS Compliance Review File do not indicate the actual actions taken after the CRs.
Table 3-4 shows, for each action type group, the number of carriers in the group that received CRs in FY 2005, the pre-CR average crash rate, the adjusted post-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-4 also shows, for each action type group, the estimated number of crashes avoided as a result of the CRs.
| Type of Planned Course of Action |
Number of Carriers with CRs in FY 2005 |
Pre-CR Average Crash Rate* |
Post-CR Average Crash Rate* |
Percent Change in Average Crash Rate |
Estimated Number of Crashes Avoided in FY 2005-2006 |
|---|---|---|---|---|---|
| Enforcement | 2,367 | 6.349 | 5.195 | –18.2 | 843 |
| Non-Enforcement | 6,574 | 5.542 | 4.689 | –15.4 | 1,463 |
| Total | 8,941 | 5.785 | 4.842 | –16.3 | 2,306 |
* — Crashes per 100 power units
Table 3-4 shows that it was anticipated that 2,367 (or 26.5 percent) of the 8,941 carriers that received CRs in FY 2005 would undergo enforcement actions. The "enforcement" carriers showed a crash rate reduction of 18.2 percent, compared to a 15.4 percent reduction for the "non-enforcement" carriers. The "enforcement" carriers accounted for 843, or 36.6 percent, of the crashes avoided in FY 2005-2006.
It should be noted that unlike all the other estimates in this report, these estimates were calculated without the use of the Control Group, since this variable applies only to carriers that received CRs. As explained in Section 3.1, the estimated numbers of crashes avoided in Table 3 4 were calculated in the same manner as the estimated numbers of crashes avoided in Tables 3 1, 3-2, and 3-3. The numbers in each table were prorated to sum to the estimate of 2,306 crashes avoided. The only difference is that the estimates in Table 3-4 were not calculated using the Control Group.
The pre-CR and post-CR average crash rates for the Control (i.e., non-CR) Group are actually the weighted averages of the average crash rates of the individual years, as shown by the following derivation.
The weighted average of the crash rates of two individual years is calculated by the equation:
Therefore, the weighted average of the average crash rates for the Control Group for FY 2004 and FY 2005
= Crashes in FY 2004 + Crashes in FY 2005 / Power Units in FY 2004 + Power Units in FY 2005
= Pre-CR Average Crash Rate for the Control Group
Also, the weighted average of the average crash rates for the Control Group for FY 2005 and FY 2006
= Crashes in FY 2005 + Crashes in FY 2006 / Power Units in FY 2005 + Power Units in FY 2006
= Post-CR Average Crash Rate for the Control Group
The 489,084 carriers in the Control Group had a pre-CR average crash rate of 2.088 crashes per 100 power units and a post-CR average crash rate of 2.056 crashes per 100 power units.
The percentage change in the average crash rate of the Control Group was calculated as follows:
Post-CR Average Crash Rate – Pre-CR Average Crash Rate / Pre-CR Average Crash Rate X 100
= 2.056 – 2.088 / 2.088 X 100
= –1.53% (i.e., a decrease of 1.53 percent)
This decrease in the average crash rate of the Control Group, and therefore, the general carrier population, is the sum of the effects of (1) any change in the average crash rate of the general carrier population and (2) other underlying factors in the general carrier population (e.g., changes in crash reporting). To determine how much of the decrease was due to each element, the change in the average crash rate of the general carrier population was calculated.
To verify if the crash rate actually decreased during the period in which the benefits from the CRs conducted in FY 2005 would have occurred (i.e., FY 2005-2006), data independent of the state-reported crash data used in the CR Effectiveness Model were used to calculate the large truck crash rates for the periods FY 2004-2005 and FY 2005-2006. The percentage change in the two crash rates was then calculated.
These crash rates were calculated using large truck crash data from the Fatality Analysis Reporting System (FARS) and the General Estimates System (GES), which are maintained by the National Highway Traffic Safety Administration (NHTSA). Counts of fatal crashes were obtained from the FARS, which contains data on a census of fatal crashes. Counts of injury crashes and property-damage-only crashes were obtained from the GES, which produces crash estimates from a national probability sample of all police-reported crashes. Crashes are included in the sample whether or not they are reported by the states to the FMCSA.
The NHTSA crash classification system differs from the National Governors’ Association (NGA) standard used by the states to report crashes to the FMCSA. In both systems, a fatal crash is defined as a crash resulting in at least one fatality, although the NHTSA rule specifically requires that at least one death occur within 30 days of the crash. For non-fatal crashes, the differences are much greater.
The NGA categories of non-fatal crashes are injury and towaway:
The NHTSA categories of non-fatal crashes are injury and property-damage-only:
The NHTSA non-fatal crash categories include many more crashes of lower severity than do the NGA non-fatal crash categories. Since it is the change in crash rates that is being measured, rather than the crash rates themselves, using the FARS and GES data should provide a reasonable indication of the change in the NGA crash rate calculated using the FMCSA's MCMIS data.
While FARS data for FY 2004, 2005, and 2006 were obtained, GES data were not available by fiscal year at the time that this analysis was performed. Thus, calendar year GES crash data were used in the model.
Power unit data were obtained from the Federal Highway Administration (FHWA). The FHWA collects truck registration data from the 50 states and the District of Columbia. The data obtained were the numbers of large trucks registered in the U.S. in CY 2003, 2004, and 2005. These CY numbers were used because (1) they are the only national registration figure available and (2) some states report their data on a fiscal year basis. Therefore, the FHWA numbers are not pure calendar year numbers, but a mixture of calendar and fiscal year numbers.
The change in the average crash rate of the general carrier population, as measured by the FARS and GES data, is calculated as follows:
Percent Change in Average Crash Rate = Post-CR Average Crash Rate – Pre-CR Average Crash Rate / Pre-CR Average Crash Rate X 100
The pre-CR crash rate is the average crash rate for the entire pre-CR period, i.e., FY 2004-2005, while the post-CR crash rate is the average crash rate for the entire post-CR period, i.e., FY 2005-2006. The pre-CR and post-CR average crash rates are calculated as follows:
Pre-CR Average Crash Rate = Crashes in FY 2004 + Crashes in FY 2005 / Large Trucks Reg. in CY ‘04 + Large Trucks Reg. in CY '05 X 100
Post-CR Average Crash Rate = Crashes in FY 2005 + Crashes in FY 2006 / Large Trucks Reg. in CY ‘05 + Large Trucks Reg. in CY ‘06 X 100
The general carrier population had a pre-CR average crash rate of 4.942 crashes per 100 power units and a post-CR average crash rate of 4.578 crashes per 100 power units.
The percentage change in the average crash rate of the general carrier population was calculated as follows:
Post-CR Average Crash Rate – Pre-CR Average Crash Rate / Pre-CR Average Crash Rate X 100
= 4.578 – 4.942 / 4.942 X 100
= –7.37% (i.e., a decrease of 7.37 percent)
Thus, the combined data from the NHTSA and FHWA suggest that the actual change in the crash rate for large trucks from FY 2004-2005 to FY 2005-2006 was a decrease of 6.22 percent.
Therefore, the increase in the crash rate of the Control Group caused by changes in other underlying factors (e.g., changes in crash reporting) in the general carrier population was:
= Percentage Change in Average Crash Rate of Control Group (from state-reported data)
- Percentage Change in Average Crash Rate of General Carrier Population (from FARS and GES data)
= –1.53% – (–7.37%)
= 5.84%
Therefore, the 1.53 percent decrease in the average crash rate of the control group, and therefore, the general carrier population, was the sum of a 7.37 percent decrease in the crash rate of the general carrier population and a 5.84 percent increase due to other underlying factors in the general carrier population (e.g., changes in crash reporting).
1 Reports documenting these results are available at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=23..
2 SafeStat (Safety Status Measurement System) is an automated, data-driven analysis system that is designed to incorporate on-road safety performance information and enforcement history with on-site compliance review information in order to measure the relative safety fitness of interstate motor carriers. A thorough description of SafeStat methodology can be found in: John A. Volpe National Transportation Systems Center, Motor Carrier Safety Assessment Division, DTS-47, SafeStat, Motor Carrier Safety Status Measurement System, Methodology: Version 8.6, January 2004. This document is available at ai.fmcsa.dot.gov/CarrierResearchResults/PDFs/SafeStat_method.pdf.
3 Reports documenting these results are available at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=23.
4 SafeStat (Safety Status Measurement System) is an automated, data-driven analysis system that is designed to incorporate on-road safety performance information and enforcement history with on-site compliance review information in order to measure the relative safety fitness of interstate motor carriers. A thorough description of SafeStat methodology can be found in: John A. Volpe National Transportation Systems Center, Motor Carrier Safety Assessment Division, DTS-47, SafeStat, Motor Carrier Safety Status Measurement System, Methodology: Version 8.6, January 2004. This document is available at ai.fmcsa.dot.gov/CarrierResearchResults/PDFs/SafeStat_method.pdf.
5 The pre-CR average crash rate is actually the weighted average of the average crash rates for FY 2004 and FY 2005. The post-CR average crash rate is actually the weighted average of the average crash rates for FY 2005 and FY 2006. A detailed derivation of these formulas can be found in Appendix A.
6 A fatal crash results in at least one fatality. An injury crash results in no fatalities, but bodily injury to at least one person who, as a result of the injury, immediately receives medical treatment away from the scene of the crash. A towaway crash results in no fatalities or injuries requiring transport for immediate medical attention, but in one or more motor vehicles incurring disabling damage as a result of the crash, requiring the vehicle(s) to be transported away from the scene by a tow truck or other motor vehicle.
7 Reports documenting these results are available at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=23.
8 A report documenting these results is available at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=23.